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Washington,  DC  20548 


May  18,  2009 


The  Honorable  Solomon  Ortiz 
Chairman 

Subcommittee  on  Readiness 
Committee  on  Armed  Services 
House  of  Representatives 

Subject:  Questions  for  the  Record  Related  to  the  Implementation  of  the  Department 
of  Defense’s  National  Security  Personnel  System 

It  was  a  pleasure  to  appear  before  your  subcommittee  on  April  1,  2009,  to  discuss  the 
Department  of  Defense’s  (DOD)  implementation  of  its  new  human  capital  system  for 
managing  civilian  personnel — the  National  Security  Personnel  System  (NSPS).1  This 
letter  responds  to  your  request  that  I  provide  answers  to  questions  for  the  record 
from  the  hearing.  The  questions,  along  with  my  responses,  follow. 


Questions  from  Chairman  Ortiz 

1.  GAO  noted  that  NSPS  was  implemented  too  quickly. 

•  What  steps  could  have  been  taken  to  roll  out  NSPS  in  a  more  orderly 
and  fair  fashion? 

As  we  have  previously  reported,  we  support  the  need  to  expand  broad  banding 
approaches  and  pay-for-performance-based  systems  in  the  federal  government.2 
However,  moving  too  quickly  or  prematurely  to  implement  such  programs,  whether 
at  DOD  or  elsewhere,  can  significantly  raise  the  risk  of  doing  it  incorrectly.  Hasty 
implementation  could  also  set  back  the  legitimate  need  to  move  to  a  more 
performance-  and  results-based  system  for  the  federal  government  as  a  whole.  Thus, 
while  it  is  imperative  that  we  take  steps  to  better  link  employee  pay  to  performance 
across  the  federal  government,  how  it  is  done,  when  it  is  done,  and  the  basis  on 
which  it  is  done  can  make  all  the  difference  in  whether  or  not  such  efforts  are 
successful.  In  our  view,  one  key  need  is  to  modernize  performance  management 


1  GAO,  Human  Capital:  Improved  Implemeyitat.ion  of  Safeguards  and  an  Action  Plan  to  Address 
Employee  Concerns  Could  Increase  Employee  Acceptance  of  the  National  Securi  ty  Personnel  System, 
GAO-09-464T  (Washington,  D.C.:  April  1,  2009). 

2  GAO,  Defense  Transformation:  DOD’s  Proposed  Civilian  Personnel  System  and  Governmentwide 
Human  Capital  Reform,  GAO-03-741T  (Washington,  D.C.:  May  1,  2003). 
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systems  in  executive  agencies  so  that  they  are  capable  of  adequately  supporting  more 
performance-based  pay  and  other  personnel  decisions. 

While  our  previous  work  does  not  prescribe  a  process  and  time  frames  for  rolling  out 
systems  such  as  NSPS,  we  have  stressed  that  agencies  should  have  an  institutional 
infrastructure  in  place  that  would  include,  at  a  minimum,  (1)  a  human  capital 
planning  process  that  integrates  the  agency’s  human  capital  policies,  strategies,  and 
programs  with  its  program  goals  and  mission  and  desired  outcomes;  (2)  the 
capabilities  to  effectively  develop  and  implement  a  new  human  capital  system;  and 
(3)  the  existence  of  a  modern,  effective,  and  credible  performance  management 
system  that  includes  adequate  safeguards,  including  reasonable  transparency  and 
appropriate  accountability  mechanisms,  to  ensure  the  fair,  effective,  and 
nondiscriminatory  implementation  of  a  new  system.  Prior  to  NSPS  implementation, 
we  cautioned  that,  while  the  DOD  leadership  had  the  intent  and  the  ability  to 
implement  the  needed  infrastructure,  it  did  not  have  the  necessary  infrastructure  in 
place  across  the  department. 

Further,  our  work  has  continued  to  stress  the  importance  of  incorporating  internal 
safeguards  into  the  design  and  implementation  of  large-scale  pay-for-performance 
programs.  In  2008,  we  evaluated  DOD’s  efforts  to  implement  nine  safeguards  and 
accountability  mechanisms. 3  We  found  that,  while  DOD  had  taken  some  steps  to 
implement  internal  safeguards  to  ensure  that  NSPS  is  fair,  effective,  and  credible,  the 
implementation  of  some  safeguards  could  be  improved.  First,  DOD  does  not  require  a 
third  party  to  analyze  rating  results  for  anomalies  prior  to  finalizing  employee  ratings, 
and  therefore  it  is  unable  to  ensure  that  ratings  are  fair  and  nondiscriminatory  before 
they  are  finalized.  Second,  the  process  has  lacked  transparency  until  recently  because 
DOD  did  not  require  commands  to  publish  final  rating  distributions,  though  doing  so 
was  recognized  as  a  best  practice  by  NSPS  program  officials  at  all  four  components. 
In  2008,  the  department  revised  its  NSPS  regulations  and  guidance  to  require 
commands  to  publish  the  final  overall  rating  results.  Third,  NSPS  guidance  may 
discourage  rating  officials  from  making  meaningful  distinctions  in  employee  ratings 
because  it  indicated  that  the  majority  of  employees  should  be  rated  at  the  “3”  level, 
on  a  scale  of  1  to  5,  resulting  in  a  hesitancy  to  award  ratings  in  other  categories.  We 
continue  to  believe  that  improved  implementation  of  these  safeguards  will  help 
bolster  employee  confidence  in  the  system  and  ensure  that  the  system  is  fair, 
effective,  and  credible. 


•  If  NSPS  continues,  what  steps  should  now  be  taken  to  move  forward? 

We  have  previously  reported  that  converting  to  NSPS  was  a  significant  transition  for 
the  department.4  We  have  further  reported  that  it  will  take  time  for  employees  to 
accept  the  system,  based  on  the  studies  conducted  by  the  Office  of  Personnel 
Management  (OPM)  on  the  federal  government  demonstration  projects  for 
performance  management.  First,  and  foremost,  DOD  needs  to  assess  and  address 
employee  engagement  in  the  system.  DOD  has  collected  survey  data  and  conducted 


3GAO,  Human  Capital:  DOD  Needs  to  Improve  Implementation  of  and  Address  Employee  Concerns 
about  Its  National  Security  Personnel  System,  GAO-08-773  (Washington,  D.C.:  Sept.  10,  2008). 

4GAO-08-773. 
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focus  groups  of  employees  under  NSPS,  but  it  is  missing  a  key  piece — an  action  plan. 
Our  2008  report  recommended  that  DOD  develop  and  implement  a  specific  action 
plan  to  address  employee  perceptions  of  NSPS  ascertained  from  feedback  avenues 
such  as,  but  not  limited  to,  DOD’s  survey  and  DOD’s  and  GAO’s  employee  focus 
groups.  At  a  minimum,  this  plan  should  include  actions  to  mitigate  employee 
concerns  about  the  potential  influence  that  employees’  and  supervisors’  writing  skills 
have  on  the  panels’  assessment  of  employee  ratings  and  the  lack  of  transparency  and 
understanding  of  the  pay  pool  panel  process.  Such  a  plan  would  demonstrate  to 
employees  that  the  department  is  listening  to  their  concerns  and  making  plans  to 
address,  as  appropriate,  the  concerns  that  are  identified.  In  short,  DOD  needs  to  tell 
the  employees  that  they  are  going  to  take  action  on  their  concerns.  In  addition,  the 
recently  announced  study  by  DOD  and  OPM  is  an  opportunity  to  assess  the  status  of 
the  system.  While  the  review  intends  to  include  a  thorough  examination  of  all  NSPS 
policies,  regulations,  and  practices,  we  would  like  to  see  DOD  leverage  this 
opportunity  to  assess  for  itself  how  the  department  is  implementing  internal 
safeguards.  Specifically,  we  are  interested  in  an  update  of  how  the  safeguards  have 
been  incorporated  into  their  policies  and  how  the  safeguards  are  working. 


2.  The  Deputy  Secretary  of  Defense,  in  cooperation  with  the  Office  of 
Personnel  Management  recently  announced  that  the  department  would 
halt  conversions  of  DOD  civilian  employees  to  NSPS,  pending  the  outcome 
of  a  review  by  DOD  and  OPM.  The  proposed  review  will  assess  whether  or 
not  NSPS  is  meeting  its  objectives  of  being  a  fair,  transparent,  and 
effective  personnel  system.  Finalizing  the  details  of  such  a  review’s 
overall  framework,  scope,  timeline,  and  leadership  will  take  time. 

•  As  DOD  and  OPM  leadership  hold  discussions  to  determine  the 

overall  framework,  scope,  and  timeline  of  the  review,  what  guidance 
or  suggestions  would  you  give  to  DOD  and  OPM  to  include  in  the 
methodology  of  this  study? 

As  we  have  previously  reported,  the  extent  to  which  DOD  incorporates  internal 
safeguards  into  the  design  and  implementation  of  NSPS  and  how  it  addresses 
employee  perceptions  of  NSPS  are  key  to  the  success  of  the  system.0  Moving  forward, 
as  DOD  and  OPM  embark  on  a  study  of  NSPS  and  review  how  NSPS  operates  and  its 
underlying  policies,  DOD  has  a  unique  opportunity  to  consider  our  previous 
recommendations,  as  well  as  all  of  the  other  internal  safeguards  key  to  ensuring  that 
performance  management  systems  in  the  government  are  fair,  effective,  and  credible. 
In  addition  to  a  review  of  internal  safeguards,  this  study  provides  DOD  the 
opportunity  to  look  at  employee  engagement  in  the  process  and  develop  an  action 
plan  to  address  employee  concerns  about  NSPS.  As  we  approached  our  work,  we 
used  a  methodology  that  systematically  took  into  account  employee  input  from  all 
levels.  We  used  a  combination  of  survey  analysis,  interviews,  and  employee 
discussion  groups  to  obtain  information  on  employee  perceptions.  In  general,  the 
combination  of  employee  surveys  with  interviews  or  discussion  groups  is  helpful 
because  it  yields  useful  information  at  the  population  level,  as  well  as  the  individual 
employee  experience  level. 


5GAO-08-773. 
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3.  One  concern  expressed  by  employees  who  have  converted  from  GS  to 
NSPS  is  that  there  is  no  real  career  progression.  Under  the  GS  system,  an 
employee  steadily  moves  up  through  the  various  grades  and  can  actually 
monitor  actual  career  progression.  There  appears  to  be  no  such  similar 
movement  in  NSPS;  an  employee,  while  receiving  pay  increases  and 
bonuses,  may  remain  in  the  same  pay  band  for  his/her  entire  career. 

•  If  this  is  a  valid  concern,  how  can  it  be  addressed,  if  NSPS 
continues? 


First,  DOD  needs  to  collect  more  information  on  what  the  issues  are  surrounding  this 
employee  perception  on  career  progression,  including  the  underlying  causes  and  the 
extent  of  this  concern,  so  that  the  department  can  determine  if  it  is  indeed  a  valid 
concern.  For  example,  is  there  an  issue  with  lack  of  career  progression  or  are 
employees  perceiving  that  there  is  an  issue  as  a  result  of  lack  of  communication  or 
education  on  the  new  system?  In  our  2008  report,  we  recommended  that  the 
department  develop  and  implement  a  specific  action  plan  to  address  employee 
perceptions  of  NSPS  ascertained  from  feedback  avenues  such  as,  but  not  limited  to, 
DOD’s  survey  and  DOD’s  and  GAO’s  employee  focus  groups.  We  believe  that  this  is 
another  example  of  how  the  department  could  use  such  an  action  plan  to  guide  its 
approach  for  addressing  employee  concerns.  Specifically,  the  plan  may  incorporate 
various  communication  and  education  strategies  to  help  employees  understand  how 
the  shift  from  pay  grades  to  pay  bands  still  affords  them  opportunities  for 
professional  development,  as  well  as  movement  through  the  pay  band.  While  we 
acknowledge  that  change  takes  time  to  gain  employee  acceptance  and  that  the 
implementation  of  NSPS  is  a  large-scale  organizational  transformation,  employee 
concerns,  such  as  these,  must  be  heard  and  addressed  accordingly  in  order  to  ensure 
greater  employee  acceptance  and,  ultimately,  successful  implementation  of  the  NSPS 
performance  management  system. 


Questions  from  Representative  Forbes 

Based  on  the  GAO  testimony,  one  safeguard  GAO  believes  needs  to  be 
implemented  to  increase  employee  confidence  in  the  pay  for  performance 
system  is  for  DOD  to  have  a  third  party  analyze  the  pay-pool 
recommendations  for  “anomalies”  before  any  final  decision  is  made  to 
determine  whether  an  employee’s  rating  accurately  reflects  the  employee’s 
performance  and  whether  any  non-merit  based  factors  contributed  to  the 
“anomaly.” 


1.  Explain  how  you  see  this  third  party  analysis  working. 

Given  that  each  agency  has  its  own  set  of  unique  challenges  and  its  own  approach  for 
handling  those  challenges,  we  believe  that  the  department  is  in  the  best  position  to 
determine  how  to  appropriately  design  and  implement  a  predecisional  analysis  for 
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NSPS.  That  said,  we  believe  that  the  third-party  analysis  should  be  conducted  by  an 
independent  reviewing  office,  such  as  a  human  capital  office,  that  is  able  to  conduct 
the  analysis  outside  of  the  chain  of  command.  Taking  the  analysis  outside  of  the 
chain  of  command  helps  to  ensure  that  the  process  remains  as  independent  as 
possible.  Seeing  that  DOD  currently  has  over  200,000  civilian  employees  under  NSPS, 
the  department  could  consider  phasing  in  the  third-party  analysis  by  starting  with  a 
representative  sample  of  employees.  A  phased  implementation  approach  recognizes 
that  different  components  of  agencies  will  often  have  different  levels  of  readiness  and 
different  capabilities  to  implement  new  authorities.  Moreover,  a  phased  approach 
allows  for  learning  so  that  appropriate  adjustments  and  midcourse  corrections  can  be 
made  before  new  policies  and  procedures  are  fully  implemented  organizationwide. 


2.  In  your  view,  who  would  the  third  party  be,  a  DOD  entity  or  a  non-DOD 
entity? 

As  noted  in  our  response  to  question  1,  we  would  expect  the  third  party  to  be  a  DOD 
entity  that  is  removed  from  the  chain  of  command — that  is,  the  human  capital  office 
or  an  office  of  opportunity  and  inclusiveness. 


3.  What  criteria  does  GAO  see  as  constituting  an  anomaly? 

Generally,  an  anomaly  would  be  characterized  as  a  set  of  ratings  for  which  there  is  a 
statistically  significant  difference  in  comparison  to  the  larger  group.  For  example,  if 
the  data  indicated  that  there  was  a  statistically  significant  difference  between  the 
ratings  of  a  particular  subset  of  the  larger  workforce  compared  to  the  larger  group  at 
that  same  level,  this  could  constitute  an  anomaly.  The  presence  of  an  anomaly  is  not 
alone  proof  that  there  is  a  problem.  Rather,  identifying  an  anomaly  in  the  data  prior  to 
finalizing  the  rating  decisions  would  enable  management  to  investigate  the  situation 
and  determine  whether  the  results  are  justified  and  merit-based. 


4.  In  investigating  “blatant  discrimination”  or  “egregious  decisions”  would 
the  employee  be  contacted  and  interviewed? 

The  predecisional  reviews  are  to  help  achieve  consistency  in  the  performance 
management  process  and  provide  reasonable  assurance  that  the  performance 
decisions  are  merit-based  and  fair.  Due  to  the  nature  of  the  investigation,  the 
employee  would  not  be  contacted  during  an  investigation.  However,  information 
provided  by  employees,  such  as  the  self-assessment,  can  be  considered  during  the 
review  process,  as  could  information  provided  by  responsible  managers  regarding 
underlying  reasons  for  any  anomalies. 


5.  Would  a  single  third  party  be  evaluating  all  1,600  pay  pools  across  DOD  to 
get  a  DOD  wide  view  of  anomalies,  or  would  1,600  third-party  reviews  be 
conducted  at  each  pay  pool  without  regard  for  a  comprehensive  DOD 
look? 
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As  noted  in  our  response  to  question  1,  each  agency  has  its  own  set  of  unique 
challenges  and  its  own  approach  for  handling  those  challenges.  As  a  result,  we  feel 
that  the  department  is  in  the  best  position  to  determine  how  to  appropriately  design 
and  implement  a  predecisional  analysis  for  NSPS.  One  approach,  as  noted  in  our 
response  to  question  1,  would  be  for  DOD  to  phase  in  the  third  party  analysis  by 
starting  with  a  representative  sample  of  employees.  Such  an  approach  recognizes  that 
different  levels  of  readiness  and  different  capabilities  exist  among  agency 
components  and  allows  for  learning  so  that  appropriate  adjustments  and  midcourse 
corrections  can  be  made  before  full  implementation. 


6.  What  effect  would  the  third-party  analyses  have  on  the  timeliness  of  the 
pay-pool  process? 

It  would  likely  add  time  to  the  existing  process.  However,  we  believe  that  it  is 
important  that  DOD  take  steps  to  ensure  that  its  employees’  ratings  are  perceived  as 
fair  reflections  of  their  performance.  Taking  additional  time  to  complete  a 
predecisional  analysis  is  one  safeguard  that  DOD  can  implement  to  raise  employee 
confidence  in  the  fairness  and  credibility  of  the  system. 


7.  Would  the  pay  pool  decisions  on  all  the  other  employees  in  the  pay  pool 
be  held  up  until  the  “anomaly”  was  resolved? 

Given  that  the  predecisional  review  is  intended  to  take  place  prior  to  the  ratings 
being  finalized  but  before  they  are  certified  and  released  to  employees,  all  other 
ratings  would  not  be  released  until  the  predecisional  review  was  completed  and 
appropriate  responses  (which  could  include  inaction)  were  determined  for  any 
anomalies  identified. 


8.  How  would  GAO  see  the  anomaly  being  corrected  -  a  directive  to  the  rater 
to  change  the  rating,  or  some  disciplinary  action  against  the  rater,  or 
some  other  form  of  corrective  action? 

Where  managers  provide  information  that  explains  the  merit-based  factors  and 
reasons  for  the  anomalies,  the  managers  would  not  change  the  ratings.  On  the  other 
hand,  managers  could  determine  that  some  vital  information  was  not  considered  that 
would  provide  a  basis  for  changing  the  rating.  In  all  cases,  it  is  the  unit  manager,  not 
the  third  party  conducting  the  predecisional  review,  that  would  determine  whether  a 
change  would  be  warranted.  Further,  the  review  is  not  intended  to  change  the  results 
to  portray  an  “ideal”  distribution,  or  to  alter  the  outcome  of  the  performance 
management  process.  The  purpose  of  the  predecisional  review  is  to  identify  if 
anomalies  exist  and,  if  found,  inform  managers  of  the  need  for  further  review  to 
provide  reasonable  assurance  that  the  basis  for  each  rating  is  fair,  credible,  and  merit- 
based. 
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9.  Does  GAO  see  any  appeal  rights  for  the  manager  or  employee  involved  in 
the  “anomaly”? 

Although  the  third-party  reviewer  identifies  the  anomalies,  it  is  the  responsible 
manager  that  examines  the  basis  underlying  the  ratings  and  is  held  accountable  for 
ensuring  the  ratings  are  merit  based.  While  it  is  unlikely,  should  a  manager  be 
disciplined,  he  or  she  might  have  appeal  rights,  depending  on  the  nature  of  the 
discipline  imposed. 

Any  employee  has  a  right  to  appeal  his  or  her  final  rating.  However,  these  appeal 
rights  do  not  apply  to  the  predecisional  review  process  since  it  is  intended  to  take 
place  prior  to  the  ratings  being  finalized  but  before  they  are  certified  and  released  to 
employees. 


For  additional  information  on  our  work  on  human  capital  issues  at  DOD,  please 
contact  me  on  202-512-3604  or  farrellb@gao.gov. 


Brenda  S.  Farrell 
Director,  Defense  Capabilities 
and  Management 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  Web  site, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngcl@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 
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